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Objectives 


The  purpose  of  this  research  is  to  develop  a  dynamic  simulation  approach  to  per¬ 
form  efficient  large  scale  simulations  of  full  aircraft  configurations,  with  the  capabil¬ 
ity  of  including  six-degrees-of-freedom  (6-DOF)  corresponding  to  realistic  aeroelastic 
motions.  While  there  has  been  some  progress  in  the  aeroelastic  problem  in  previ¬ 
ous  efforts,  the  flow  component  has  been  crudely  modeled  using  either  Euler  inviscid 
description  or  the  Reynolds  averaged  Navier-Stokes  equations,  both  of  which  have 
proven  to  be  inaccurate  and  thus  inadequate  for  the  Air  Force  needs.  The  emphasis 
of  the  completed  work  has  been  on  capturing  very  accurately  the  vorticity  generation 
process  and  the  unsteady  three-dimensional  separation  phenomena  on  the  body  by 
resolving  all  energetic  flow  scales  in  the  near- wall  and  near- wake  regions.  Away  from 
the  body  a  subgrid  model  will  be  used  to  account  for  the  reduced  resolution.  A  new 
simulation  paradigm  has  been  developed  for  such  flows  employing  the  concept  of  dy¬ 
namic  simulation  based  on  surgical  p-refinement  for  transitional  and  turbulent  flows 
at  high  speed,  where  significantly  energetic  flow  scales  are  resolved  dynamically  as 
they  are  convected  downstream  or  as  new  structures  are  generated  and  shed  off  from 
the  body. 


Major  Accomplishments 

•  Development  of  a  new  projection  formulation  of  hyperbolic  and  parabolic 
problems  using  a  discontinuous  Galerkin  formulation:  These  new  projections 
were  used  to  develop  a  high-order  finite- volume  type  formulation  for  high-speed  flows 
that  is  conservative  and  does  not  require  the  use  of  any  limiters.  For  smooth  regions 

a  high-order  representation  is  employed,  whereas  for  regions  containing  strong  shocks 
adaptive  h-refinement  is  employed  recovering  the  standard  finite  volume  formula¬ 
tion.  This  is  the  first  ever  formulation  that  provides  high-order  accuracy  on  standard 
unstructured  and  viscous  grids  while  maintaining  element-wise  conservativity  and 
monotonicity,  which  are  key  features  for  simulating  accurately  high-speed  flows. 

•  Development  of  an  Arbitrary  Lagrangian  Eulerian  (ALE)  formulation 
for  moving  and  time-dependent  domains:  Specifically,  the  ALE  formulation 
has  been  implemented  within  the  Discontinuous  Galerkin  framework.  It  allows  the 
computations  of  flows  around  moving  and  deforming  bodies.  A  novel  development 
is  an  efficient  algorithm  for  mesh  movement  around  deforming  bodies  that  does  not 
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require  matrix  inversion.  In  fact,  the  entire  formulation  is  matrix-free. 

•  Development  of  dynamic  h-refinement  of  the  mesh  around  discon¬ 
tinuities:  In  regions  of  large  gradients,  high-order  elements  are  subdivided  into  a 
set  of  smaller  triangles  in  two-dimensions  or  tetrahedra  in  three-dimensions.  Stable 
upwind  schemes  are  applied  on  these  smaller  elements.  If  the  gradient  decreases, 
these  elements  are  interpolated  back  to  the  polynomial  space.  This  h-p  adaptive  al¬ 
gorithm  allows  great  flexibility  in  computing  flows  with  discontinuities  (supersonic 
flows)  without  loss  of  accuracy. 

•  De- Aliasing  on  Unstructured  Grids:  Aliasing  errors  can  produce  erroneous 
results  in  turbulence  simulations  and  in  any  other  long-term  integration  of  nonlinear 
unsteady  problems.  While  de-aliasing  rules  for  Fourier  methods  have  been  established 
and  shown  to  be  very  effective  (e.g.  the  3/2  rule),  there  are  currently  no  procedures 
to  remove  aliasing  errors  on  non-uniform  grids.  We  have  developed  the  first  such 
effective  procedure  and  apply  it  to  DNS  of  flows  past  bluff  bodies.  Specifically,  the 
algorithm  consists  of  a  super-collocation  procedure  followed  by  a  Galerkin  projection. 
The  typical  number  of  extra  quadrature  points  is  twice  the  number  of  degrees  of  free¬ 
dom  (for  the  discretization  of  nonlinear  terms  only).  We  have  performed  tests  with 
computational  complexity  equivalent  to  the  3/2  rule,  and  the  results  are  very  good. 

•  The  Spectral  Vanishing  Viscosity  (SVV)  Method.  This  formulation  is 
based  on  a  rigorous  theoretical  framework:  It  guarantees  monotonicity  of  the  solution 
while  it  preserves  the  fast  exponential  convergence  of  spectral  discretization.  This  re¬ 
sult,  which  was  first  obtained  for  nonlinear  conservation  laws,  leads  to  high  accuracy 
and  great  efficiency  in  LES  as  it  avoids  filtering  and  excessive  dissipation.  Specifi¬ 
cally,  the  Navier-Stokes  equations  are  enhanced  with  a  viscous  convolution  operator 
described  by  a  mode-dependent  viscosity  which  acts  only  on  high  modes  unlike  the 
classical  eddy-viscosity  models  that  modify  the  entire  spectrum  including  the  low- 
wave  number  energetic  scales.  It  resembles  hyperviscosity  which  is  often  used  with 
success  in  homogeneous  turbulence  simulations  in  order  to  achieve  high  Reynolds 
number  turbulence.  However,  the  important  difference  is  that  in  the  SVV  approach 
all  parameters  are  given  by  the  theory,  and  also  the  convolution  operator  is  second- 
order,  which  means  that  it  can  be  used  in  conjunction  with  finite  element  and  finite 
volume  techniques  unlike  the  hyperviscous  kernels. 

Specific  tasks  we  performed  included  parametric  studies  and  comparisons  with 
standard  DNS  benchmarks  revealing  a  two-orders  of  magnitude  savings  in  CPU  sim- 
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ulations,  and  high  Reynolds  number  simulations  past  a  cylinder  {Re  =  160, 000,  for 
first  time). 

•  Simulation  and  VR-Visualization  of  a  full  F15  configuration.  This  is 
the  first  time  that  such  simulations  are  performed  on  a  Linux  cluster  of  PCs!  The 
simulation  employed  about  150,000  tetrahedra  elements  describing  accurately  the 
geometry  of  the  F15  with  fourth-order  Jacobi  polynomials.  The  SVV-LES  approach 
described  above  was  used  in  the  simulations.  The  average  CPU  time  per  time  step  on 
a  cluster  of  20  PC/Linux  (Road  Runner  in  University  of  New  Mexico)  cluster  was  30 
seconds.  This  simulation  demonstrates  that  very-large-scale  simulations  of  interest 
to  AFOSR  will  be  possible  in  the  near-future  on  relatively  small  clusters  of  PCs. 

In  addition  to  the  simulation  of  the  unsteady  flow,  3D  visualizations  in  the  CAVE 
(virtual  reality  facility)  have  been  developed  and  applied  to  the  F15  simulation  data. 
Dr.  Sakell  (the  AF  PM  of  this  project)  was  invited  to  view  and  interact  with  this  data 
in  this  new  and  potentially  breakthrough  simulation  and  visualization  environment. 
The  PI  has  been  collaborating  with  computer  scientists  to  develop  novel  interaction 
and  visualization  tools  for  external  flows,  using  a  kit  of  virtual  tools,  including  rakes  of 
streamlines,  particles,  vector,  pressure  iso-surfaces,  vorticity  and  other  aerodynamic- 
specific  quantities.  Particular  attention  has  been  paid  to  visualizing  in  real  time  both 
the  global  (entire)  field  as  well  as  details  of  the  flow,  e.g.  horseshoe  vortex  around 
the  rudder.  This  was  accomplished  using  the  intrinsic  hierarchy  in  modal  space  of 
the  spectral/hp  element  method  that  the  PI  and  his  students  have  developed  with 
the  sponsorship  of  AFOSR. 


Personnel 

•  Faculty:  G.E.  Karniadakis,  Professor  of  Applied  Mathematics 

•  Collaborators  and  PhD  Students:  Prof.  S.  J.  Sherwin,  Prof.  C.-H.  Su,  I.  Lomtev, 
R.M.  Kirby,  T.C.  Warburton,  V.  Symeonidis,  D.  Xiu. 


Publications 

1.  I.  Lomtev,  “A  discontinuous  Galerkin  method  for  the  compressible  Navier- 
Stokes  equations  in  stationary  and  moving  3D  domains”,  PhD  thesis.  Brown 
University,  (completed  in  1999;  supervised  by  the  PI). 
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2.  G.-S.  Karamanos  and  G.E.  Karniadakis,  “A  spectral  vanishing  viscosity  method 
for  large  eddy  simulations”,  J.  Comp.  Phys.,  vol.  162,  pp.  22-50,  2000. 

3.  C.  Evangelinos,  S.J.  Sherwin  and  G.E.  Karniadakis,  “Parallel  DNS  algorithms 
on  unstructured  grids”.  Comp.  Meth.  Appl.  Mech.  Engr.,  vol.  184,  pp. 
401-425,  2000. 

4.  I.  Lomtev,  R.M.  Kirby  and  G.E.  Karniadakis,  “DNS  for  flow  past  a  3D  flexible 
wing”.  Proceedings  of  Recent  Advances  in  DNS  and  LES,  pp.  279-290,  Kluwer, 
1999  (also  cover  of  the  book). 

5.  R.M.  Kirby,  T.C.  Warburton,  I.  Lomtev  and  G.E.  Karniadakis,  “A  discontinu¬ 
ous  Galerkin  spectral/hp  method  on  hybrid  grids”.  Applied  Numerical  Mathe¬ 
matics,  vol.  33,  pp.  393-405,  2000. 

6.  “Discontinuous  Galerkin  Methods:  Theory,  Computation  and  Applications”, 
(editors;  B.  Cockburn,  G.E.  Karniadakis  and  C.-W.  Shu),  Lecture  Notes  in 
Computational  Science  and  Engineering,  Springer,  2000. 

7.  B.  Cockburn,  G.E.  Karniadakis  and  C.-W.  Shu,  “The  development  of  discon¬ 
tinuous  Galerkin  methods”.  Proceedings  on  Discontinuous  Galerkin  Methods: 
Theory,  Computation  and  Applications”,  pp.  3-50,  Springer  2000. 

8.  C.  Evangelinos,  I.  Lomtev  and  G.E.  Karniadakis,  “Shedding  patterns  in  flow- 
structure  interactions”,  J.  Visualization,  vol.  2(2),  pp.  135-142,  1999. 

9.  G.E.  Karniadakis  and  S.J.  Sherwin,  “Spectral/hp  Element  Methods  for  CFD”, 
Oxford  University  Press,  1999. 

10.  1.  Lomtev,  R.M.  Kirby  and  G.E.  Karniadakis,  “A  discontinuous  Galerkin  method 
in  moving  domains” ,  Proceedings  on  Discontinuous  Galerkin  Methods:  Theory, 
Computation  and  Applications”,  pp.  375-383,  Springer  2000. 

11.  1.  Lomtev,  R.M.  Kirby  and  G.E.  Karniadakis,  “A  discontinuous  Galerkin  ALE 
method  for  compressible  viscous  flows  in  moving  domains”,  J.  Comp.  Phys., 
vol.  155,  pp.  128-159,  1999. 

12.  1.  Lomtev,  and  G.E.  Karniadakis,  “A  discontinuous  Galerkin  method  for  the 
Navier-Stokes  equations”,  Int.  J.  Num.  Meth.  Fluids,  vol.  29,  pp.  587-603, 
1999. 
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13.  G.E.  Karniadakis,  “Simulating  turbulence  in  complex  geometries” ,  Fluid  Dy¬ 
namics  Research,  vol.  24  (6),  pp.  343-362,  1999. 

14.  T.C.  Warburton,  S.J.  Sherwin  and  G.E.  Karniadakis,  “Basis  functions  for  tri¬ 
angular  and  quadrilateral  high-order  elements”,  SIAM  J.  Scientific  Computing, 
vol.  20(5),  pp.  1671-1695,  1999. 

15.  T.C.  Warburton,  I.  Lomtev,  Y.  Du,  S.J.  Sherwin,  and  G.E.  Karniadakis,  “Galerkin 
and  discontinuous  Galerkin  spectral/hp  methods”.  Comp.  Meth.  Appl.  Mech. 
Engr.,  vol.  175,  pp.  343-359,  1999. 

16.  “Spectral,  Spectral  Element  and  hp  Methods  for  CFD” ,  guest  editor  of  Comp. 
Meth.  Appl.  Mech.  Engr.,  (co-editors:  M.  Ainsworth  and  C.  Bernardi),  vol. 
175,  1999. 

17.  G.S.  Karamanos,  C.  Evangelinos,  R.  Boes,  R.M.  Kirby  and  G.E.  Karniadakis, 
“DNS  of  turbulence  on  a  PC/Linux  cluster:  Fact  or  Fiction?”,  Proc.  Super- 
Computing  1999. 

18.  R.M.  Kirby,  T.C.  Warburton,  S.J.  Sherwin  and  G.E.  Karniadakis,  “The  NEK- 
TAR  code:  Dynamic  simulations  without  refinement”.  Second  International 
Symposium  on  Computational  Technologies  for  Fluid/Thermal/Chemical  Sys¬ 
tems  with  Industrial  Applications,  August  1-5,  Boston  MA,  1999. 


Interactions/Transitions 

The  PI  organized  (with  B.  Cockburn  and  C.-W.  Shu)  the  first  International  Sym¬ 
posium  on  Discontinuous  Galerkin  Methods  on  May  24-26,  in  Newport,  RI  USA. 

The  PI  organized  a  mini-Symposium  on  “Large  Eddy  Simulations  Using  Finite 
Elements”  in  the  International  Conference  on  Finite  Elements  in  Flow  Problems  held 
in  Austin,  Texas  (May  30  -  April  4,  2000).  He  has  also  continued  his  interactions 
with  Boeing  personnel  (Dr.  D.  Young  and  Dr.  A.  Cary)  for  comparisons  between 
LES-SVV  results  and  Boeing’s  experimental  and  numerical  results. 

The  AFOSR-sponsored  research  was  presented  in  the  past  three  years  at: 
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•  DOE/Oakridge  Workshop  on  Discontinuous  Galerkin  Methods  for  Materials 
(1998) 

•  NSF  Woskshop  on  New  Computational  Challenges  (1998) 

•  AIAA  Fluid  Dynamics  Conference  on  LES  (1998) 

•  ICOSAHOM’98  Symposium  on  Corner  Singularities  (1998) 

•  Japanese  Society  of  Fluid  Mechanics  30th  Anniversary  Symposium  (1998) 

•  University  of  Tokyo,  Department  of  Mechanical  Engineering  (1998) 

•  Turkey  Workshop  on  Industrial  and  Environmental  Applications  of  DNS/LES 
(1998) 

•  Los  Alamos  National  Laboratories  (1999) 

•  First  International  Symposium  on  DCM,  Newport  RI  (1999) 

•  DOE  Workshop  on  mesh  generation,  Argonne  National  Lab  (1999) 

•  DoD  High  Performance  Computing,  Monterey  CA  (1999) 

•  Second  International  Conference  on  DNS/LES,  Rutgers  University  (1999) 

•  ASME  CFD  Symposium  on  Flow-Structure  Interactions,  Boston  (1999) 

•  National  Congress  on  Computational  Mechanics,  Boulder  Co  (1999) 

•  CISM,  Udine,  Italy  (1999) 

•  Los  Alamos  National  Laboratories  (1999)  (invited) 

•  Supercomputing’99,  Portland  (1999) 
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•  American  Physical  Society,  DFD  Meeting  (1999)  (invited/keynote) 

•  AFOSR/NSF  Workshop  on  LES  (1999)  (invited  participant) 

•  p-  and  hp  Finite  Element  Methods,  St.  Louis  (2000)  (invited) 

•  Flow  Induced  Vibration  Conference,  Switzerland  (2000) 

•  Simulation  Based  Design  Workshop,  Newport  RI/NUWC  (2000)  (invited) 

•  lUTAM  Conference  on  Wakes  &  Flow  Structure  Interactions,  Prance  (2000) 

•  DOD-HPCMP  Users  Group  Conference,  Albuqurque/NM  (2000) 

•  International  Conference  on  Finite  Elements  For  Flow  Problems  (2000) 

•  HPC  Defence  Science  Board  (2000)  (invited) 


Technology  Transfer 

The  code  NEKTAR  has  been  disributed  to  more  than  one  dozen  Universities  and 
Laboratories.  Some  of  them  include  Boeing,  Inc.,  AF  Kirkland  Lab,  MIT,  Caltech, 
UC  Berkeley,  Cornell  University,  Penn  State  University,  University  of  Wisconsin, 
Imperial  College,  Oxford  Computing  Laboratory,  University  of  Tokyo,  University  of 
Bologna,  Norwegian  University  of  Science  &  Technology,  North  Carolina  University, 
Florida  State  University,  Sandia,  OAK  Ridge  Labs,  Nielsen,  Inc.  etc.  There  is  now 
documentation  of  the  code  both  for  users  as  well  as  developers.  This  is  an  OpenSource 
code  that  runs  on  all  available  platforms  including  clusters  of  PCs. 
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